Neural Networks

Wave recurrent neural networks (wRNNs) are biologically inspired recurrent architectures that use traveling-wave dynamics to support sequence learning and memory. However, their input-to-hidden pathway remains relatively simple compared with biological neurons, where dendrites perform nonlinear input integration. In this study, we introduce the Dendritic Wave Recurrent Neural Network (DWRNN), which augments the input pathway of the wRNN with nonlinear basal dendritic branches while preserving the original recurrent wave dynamics. We evaluate DW-RNN on a simple copy task, sequential MNIST (sMNIST), permuted sequential MNIST (psMNIST), and noisy sequential CIFAR-10 (nsCIFAR-10). On the copy task, DW-RNN shows learning behavior comparable to the standard wRNN, suggesting that dendritic input integration does not disrupt the recurrent wave-based memory mechanism. On the three sequential image-classification benchmarks, DW-RNN outperforms the standard wRNN, improving accuracy from 97.27 {+/-} 0.15% to 97.82 {+/-} 0.12% on sMNIST, from 96.74 {+/-} 0.17% to 96.92 {+/-} 0.10% on psMNIST, and from 54.30 {+/-} 0.79% to 55.65 {+/-} 0.55% on nsCIFAR-10. In addition to improving mean accuracy, DW-RNN exhibits lower across-seed variability on all three classification benchmarks, suggesting that dendritic input integration may improve the stability of wRNN training. Hidden-activity visualizations further show that DW-RNN preserves the characteristic traveling-wave patterns of the original wRNN. These results suggest that dendritic computation and traveling-wave recurrent dynamics provide complementary mechanisms for biologically inspired sequence learning.

Cerebral cortical networks in the mammalian brain exhibit a non-random organization in which reciprocal projections, although widespread, are systematically asymmetric in strength: feedforward connections are consistently stronger than their feedback counterparts, particularly in sensory cortices. This "no-strong-loops" principle is thought to prevent runaway excitation and maintain stability, yet its actual computational impact remains unclear. Here, we use computational analysis and modeling to show that connectivity asymmetry supports high working-memory capacity, whereas increasing reciprocity reduces memory capacity and representational diversity in reservoir-computing models of recurrent neural networks. We systematically examine synthetic architectures inspired by mammalian cortical connectivity and find that sparse, modular, and hierarchical networks achieve superior performance, relative to random, small-world, or core-periphery graphs, but only when reciprocity is constrained. Validated on directed mammalian (macaque, marmoset, rat, and mouse) connectomes, these results indicate that restricting reciprocal motifs yields functional benefits in sparse networks, consistent with an evolutionary strategy for stable, efficient information processing in the brain. These findings suggest a biologically-inspired design principle for artificial neural systems.

The organizational principles of natural neural networks could inspire the new architecture design of artificial neural networks (ANNs). Analysis of single-neuron connectomes of mouse brains revealed distinct profiles of three-node connectivity motifs in various cortical areas and hippocampal formation. A connectome-informed neural network algorithm ("CINA") was developed to incorporate natural connectivity motifs into ANN algorithms represented by recurrent neural network (RNN) and transformer-based large language model (LLM). We found that incorporation of the average profile of cortical motifs improved the RNNs performance in noise-resistant categorization and motor learning benchmark tasks, as compared with RNNs with random connectivity. Notably, incorporating cortex-specific motifs further elevated the RNNs performance in tasks related to the cortical function, and this effect was enhanced by artificially increasing the bias in the motif profile. Similar experimental results were verified on an LLM using Motif-Transformer for natural language question answering and brain-signal decoding tasks. Graph-theoretic analyses showed that incorporating natural motifs drove the emergence of modular and small-world properties in ANNs. Together, we demonstrated not only connectome-inspired optimization of ANN architecture but also functional significance of specific motif profiles in various cortices.

Equilibrium propagation (EP) is a biologically plausible alternative to backpropagation that has demonstrated competitive performance across a range of machine learning tasks. Recent work has extended EP to spiking neural networks (SNNs), leveraging leaky integrate-and-fire (LIF) neurons and spike-based plasticity rules to improve biological realism while maintaining strong performance. In this work, we propose an EP-based SNN framework that combines LIF neural dynamics with a predictive learning rule, replacing conventional spike-timing-dependent plasticity (STDP) with a learning rule more directly aligned with predictive coding principles. We evaluate the proposed model on multiple image classification benchmarks, including MNIST, KMNIST, and Fashion-MNIST, and compare its performance with a BP-trained LIF SNN baseline. Our results show that the proposed EP-based LIF model (EP+LIF) achieves competitive accuracy across datasets, with performance approaching that of the BP-trained counterpart (BP+LIF) while relying on a biologically motivated local learning rule. In addition, analysis of hidden-layer spiking activity reveals that EP+LIF produces more persistent hidden-state activity, whereas BP+LIF yields sparser spiking representations. These results demonstrate that predictive learning can support effective EP-based training in LIF spiking networks, while also highlighting differences in activity patterns that motivate future work on activity regulation and sparse spiking dynamics.

Categorisation is often treated as a form of compression: a high-dimensional stimulus space is reduced to a smaller set of behaviourally or cognitively useful classes. However, compression alone does not determine whether a category map is useful. The present manuscript develops an information-theoretic framework for evaluating categorisation in terms of both category complexity and target-relevant information preservation. Across a set of synthetic demonstrations, alternative category maps over the same stimulus space are shown to preserve different target variables, including identity, action, nuisance, and hierarchical category structure. The framework is then extended to learned visual representations by analysing layer-derived category maps from a pretrained ResNet-50 network applied to CIFAR-10 images. Two scenarios are compared: a clean-only object run and a pooled nuisance run containing clean, blurred, pixelated, and noise-perturbed images. The results show that category maps can have substantial entropy while preserving information about a variable that is not aligned with the specified target, and that the value of a categorisation depends on the target variable to be preserved. The manuscript argues that categorisation should therefore be evaluated not only by compression or separability, but by the information retained about a specified cognitive, behavioural, or computational target.

Human object recognition is robust to challenging conditions, such as when ones view of an object is fragmented due to an occluding foreground object. In comparison, deep neural networks (DNNs) are typically more susceptible to occlusion, suggesting that human vision relies on distinct mechanisms. Here, we investigated the role of visual diet in the emergence of these mechanisms by asking whether human-like robustness might arise in DNNs when trained with image datasets that better reflect the properties of occlusion in natural vision. We trained convolutional and transformer DNNs to classify clear images only, images augmented with artificial occluders (i.e., geometric shapes) or natural occluders (objects segmented from photographs). We then evaluated DNN occlusion robustness and compared their performance profiles with 30 human participants. We found that DNNs trained with artificial occluders remained vulnerable to natural occlusion and exhibited less human-like performance than those trained with natural occlusion. Our findings suggest that human robustness to visual occlusion arises from learning to disentangle natural objects from each other rather than simply learning to recognize objects from partial views. They also imply that commonly used forms of artificial occlusion are unsuitable for the evaluation or promotion of robustness to real-world occlusion in DNNs.

The primary objective of system neuroscience is to understand the functional mapping and its causation in the dynamics of the brain network. Some experimental and methodological studies suggest that functional modularity and its hierarchical information processing in the brain network are crucial to understanding the functional role of task-specific or state-specific information flow in the brain. However, because most of the established techniques for detecting effective network structures in the neuroscience research field are strongly based on the "Granger causality" perspective, existing causal discovery methods specified for brain network analysis cannot identify the causal hierarchy in the modular network in the brain due to spurious correlation issues and indistinguishability of causal direction under the Gaussianity of observational noise in a linear system. To address the issues, we developed a causal discovery method for synchronous neural dynamics, called the Jacobian-informed linear non-Gaussian acyclic model, "j-VAR-LiNGAM", by incorporating the information of the Jacobian matrix determined from a phase-coupled oscillator model estimated from observed neural data into the VAR-LiNGAM algorithms. The method was validated by showing that it could extract causal ordering in both synthetic data and empirical neural observed data. Moreover, by analyzing the observed neural oscillatory signals obtained from mice and humans, we confirmed that our method identified causally hierarchical structures in the brain, which aligned with the neurophysiological interpretations. These findings suggested that our proposed method can reveal the neural basis of hierarchical information processing in the brain network.

Improving long-term memory in artificial neural networks remains an open challenge. To address this, we developed a novel brain-inspired framework for memory prioritization based on the principle of emotional valence. Our framework includes: (i) a valence-weighted cross-entropy loss that scales the learning signal by the valence magnitude, analogous to neuromodulation; (ii) an amygdala-inspired module that learns high-dimensional valence embeddings; and (iii) a hippocampus-inspired module that integrates valence embeddings into the attention mechanism to modulate information retrieval. We demonstrated the generalization of our framework across spatial, episodic, and language-based memory tasks, consistently improving memory prioritization and long-term retention of high-salience information. In addition to improving long-term memory, we also showed that our framework can help mitigate the "lost-in-the-middle" problem in language modeling. More generally, this research provides further evidence of the potential of brain-inspired algorithms to advance the field of machine learning.

1Hebbian-like learning has been repeatedly confirmed experimentally, yet computational models usually require non-local signals, such as backpropagating errors, to solve complex cognitive tasks. Recent cortical electron microscopy data suggests a model where synapses follow different plasticity rules depending on whether they are in a large or small state. Large synapses often include a spine apparatus, a calcium reservoir that influences synaptic dynamics and can alter rules of synaptic plasticity. Here, we test the computational outcomes of networks which compute with synapses switching their plasticity rules based on their strength. We designed a recurrent neural network (RNN) with synapses that switch between two learning rules: a Hebbian-like rule for weak synapses and a credit-assignment rule (backpropagation, BP) for strong synapses. We found that our plasticity-switching RNN (psRNN) learns cognitive tasks (e.g. working memory) in fewer trials than BP-only RNNs, despite fewer synapses using credit assignment. Three mechanisms underlie this advantage: BP samples multiple parameter configurations for better gradient estimation, Hebbian plasticity creates a dynamic task-relevant initialization, and the switching mechanism prevents Hebbian synapses from growing into unfavorable parameter regions. The interaction between rules also produces lower-rank, more feedforward recurrent structure, providing testable connectomic predictions and a framework for reconciling local learning rules observed in the brain with non-local rules used in computational models. Significance statementCredit assignment is essential for learning in biological and artificial systems. Credit assignment can be achieved via complex synaptic plasticity rules, yet biological data points to plasticity rules at most synapses being simpler. Here we ask if networks with a combination of simple and complex plasticity rules could solve biorealistic tasks. Surprisingly, replacing the computationally expensive error-based learning with simple, experience-dependent, changes in a large portion of synapses improves learning in our models. This substitution changes several computational properties of the model, offering new hypotheses for computing in the brain.

Neural system identification approaches use empirical data to fit the stimulus-response functions of neurons. Augmented by deep neural networks, such models have achieved high predictive performance and allow to perform in silico experiments to test hypotheses. Yet, many of these methods ignore common features in visual systems, such as inhibitory interactions between neurons, which are essential for nonlinear neural computation. Here, we incorporate inhibition as an inductive bias into a deep model for neural prediction and investigate the influence of inhibition on the learned transfer functions. To this end, we employ difference-of-Gaussian (subtraction) and within-channel divisive normalization (division), which have been proposed to relate inhibition to neural processing, in deep networks for predicting visual responses. We observe that incorporating such operations maintains the predictive performance and encourages the learning of biologically plausible kernels reminiscent of neural representation in early vision. Additionally, our in silico experiments demonstrate that implementing either sub-tractive or divisive operation benefits the learning of surround suppression but not cross-orientation inhibition. Interestingly, while division increases the sparsity of activation and reduces the sparsity of weights, subtraction has the reverse effect.

Navigation requires perception: location must be inferred from noisy and ambiguous egocentric sensory inputs, as in visual estimation of distance. However, many classical models of spatial representation implicitly assume that allocentric location is directly observable, thereby neglecting perceptual uncertainty. Here, we compare such a model with a Bayesian ideal observer that explicitly incorporates perceptual inference. We find that the Bayesian observers beliefs over location more accurately reproduce key properties of place cell activity, including place field width, area, and density, within and across environments. Using analytic arguments and numerical simulations, we show that recurrent neural networks trained to predict the next egocentric sensory input learn representations resembling Bayesian beliefs and yield place cell-like activity in both familiar and unfamiliar environments, outperforming autoencoders trained to reproduce the current input. Together, these results suggest that hippocampal circuits may construct Bayesian cognitive maps from experience through predictive perceptual learning.

We adapted an olfactory neuromorphic algorithm to image and sound recognition. To achieve this, we carried out specific preprocessing procedures that were tailored to each modality. For images, we used the NIST digits dataset directly. For sound, we used samples from the Google Speech Command dataset. A gammatone filter was applied to each to reduce the noise of the short audio sample and convert the temporal sound signal to a positional frequency signal. The single stimulus test algorithm was then modified to handle audio processing on extracted columns from a gammatone filter spectrogram obtained from the sound file. We also implemented PCA for all modalities, retaining around 90% of the variance. The results showed that over sequential 'olfactory' gamma cycles, the algorithm successfully achieved one-shot online learning over the image and sound modalities as well. However, PCA representations did not attain high similarities to their corresponding templates for all three modalities.

Tracking dynamic moving objects in the external world is ethologically important for many organisms. Recent experiments have examined neural dynamics supporting such behaviors by employing visually-guided pursuit in freely moving rodents, yet computational principles underlying this cognitive process are not well understood. To address this, we developed a recurrent neural network model for examining the predictive behaviors and computations that emerge during pursuit. We demonstrate that the model generates internal predictions of the targets future locations, with anticipatory behaviors increasing with exposure to stereotyped trajectories of the target. These internal predictions can be used by the model to pursue a target in a complex environment, and the models emergent strategy is aligned with behavior when tested in rodents. In investigating the computations that underlie the models ability to perform predictive pursuit, we found units sensitive to the position of the target relative to the artificial agent, a representation analogous to egocentric target neurons observed in animals performing pursuit tasks. Ablating these units significantly reduced model performance, establishing a causal role of this functional response type in efficient pursuit. Given the complexity of the task and agent behavior, we hypothesized that RNN models may use high-dimensional neural codes to support predictive pursuit. To test this, we trained models of varying rank and found that anticipatory behavior emerged only when the rank was sufficiently high, despite strong pursuit performance in lower rank models. All RNNs encoded the egocentric location of the target, whereas allocentric self and target locations emerged only in high-dimensional networks. Overall, our results suggest that, unlike commonly studied vision, motor, or memory tasks, predictive pursuit emerges in high-dimensional networks with sufficient resources.

AO_SCPLOWBSTRACTC_SCPLOWThere is a gap between neuroscientific theories of learning and the speed of learning observed in many experiments. Since the Cognitive Revolution of the 1950s, compositionality has played a central role in efforts to bridge this gap. Roughly, a compositional system is one where distinct modules are combined according to a set of rules in order to accomplish complex tasks. Recently, significant progress has been made in understanding the emergence of modules in both biological and artificial neural systems. How, and under what conditions, the rules of module recombination are represented in these systems remains an open question. Here we present a neural model that can leverage these rules to dramatically speed up learning. We first show that when faced with multiple tasks which share subcomponents, models learn a low-dimensional representation that captures how subcomponents are reused across the task set. These low-dimensional spaces encode the structure that governs how modules should be recombined. Restricting learning to these subspaces greatly reduces the amount of experience needed to acquire a novel task, even when learning from reinforcement on single trials. In some cases, we can leverage the geometric regularities of these representations to reduce learning to a form of hypothesis testing over a small set of discrete points. Finally, we use this theory to model both behavioral and neural data from non-human primates performing a compositional task, and show that key features in this data are consistent with a model in which exploration during learning is restricted to these low-dimensional spaces. Overall, this work shows that the advantages of modularity in neural systems can be greatly improved upon when models represent the structure of module reuse. Both these features working in tandem lead to learning on timescales similar to biological intelligences, and hence provide a model for how such fast, adaptable behavior can emerge from systems of neurons.

Robotic motor control built on liquid neural networks and related continuous-time models, such as LTC and CfC, is typically trained offline via backpropagation through time and lacks an explicit mechanism for recalibrating online as plant dynamics change. We ask whether a frozen CfC core, whose liquid state spans a fixed continuous-time basis, can support cerebellar-style online adaptation by adapting only its linear readout with a climbing-fiber-like error signal. In a planar two-link reaching simulation with a velocity-dependent curl force field, we adapt the readout online with a feedback-error-learning (FEL) signal under a least-mean-squares (LMS) rule, leaving the core untouched. The frozen-core readout-only controller re-straightens curl-perturbed reaches and, upon field removal, produces a mirror-image after-effect, a behavioral signature consistent with internal-model learning, which a feedback-only controller does not produce. The result generalizes from a dense CfC to a sparse Neural-Circuit-Policy (NCP) wiring when the recurrent state, rather than the projected motor output, is used as the readout basis; it is robust to force-field strength and direction; and a recursive-least-squares variant adapts faster but de-adapts slowly because its covariance collapses, a rigidity that a covariance-reset safe-forgetting rule removes. Within the explored two-link planar simulation range, we did not find a readout-only failure case that required adapting the frozen core in the tested conditions. In this simulation study, adapting only the readout therefore provides a biologically inspired, low-cost online error-adaptation layer for offline-trained continuous-time controllers.

Stabilizing synaptic plasticity together with the neurons activity has remained a central challenge in theoretical neuroscience since the introduction of Hebbian learning principles. Classical Hebbian learning rules typically lead to unbounded synaptic growth, motivating the development of stabilization mechanisms such as normalization methods, BCM-type learning, synaptic scaling and others. While these approaches can prevent divergence, they can also exhibit different limitations e.g. resulting in too-sparse synaptic configurations or leading to poor scalability with increasing network size. A recently introduced meta-plasticity mechanism, termed annealed linear learning (ALL), dynamically reduces the learning rate as neuronal output increases, thereby preserving stable and interpretable fixed-point behavior of the output. However, the original formulation leads to an irreversible decay of the learning rate, preventing adaptation to changing environmental conditions. To address this, in the present study, we balance learning rate reduction at large outputs with recovery at small outputs and in addition introduce forgetting that gradually reduces synaptic weights. These extensions allow the system to discard outdated representations and adapt to novel input conditions. Analytical investigations demonstrate that the favorable output fixed-point properties of the original ALL framework are preserved under the extended rule. Furthermore, simulations with an artificial agent show that the proposed mechanism enables robust and fast re-learning and adaptation in changing environments.

The phenomenon of splitting was originally observed in hamsters which, after prolonged exposure to constant light, exhibit two rest/wake cycles within a subjective day. Splitting is a consequence of the left and right suprachiasmatic nuclei (SCN) falling out of synchrony. While it is known that split activity is characterized by an antiphase relationship between the left and right SCN and between the core and shell within each hemisphere, the role of the commissural projections that connect the right and left SCN is not known. In the present study, we investigate the impact of the inter-hemispheric connections on the split and unsplit dynamics of a computational model of the bilateral SCN. Our model has 4 nodes corresponding to each right and left core and shell. We simulated our bilateral model under different lighting conditions and measured its period and the phase relationships among the 4 nodes. To further characterize the dynamics of the system, we performed a bifurcation analysis. We found that the bilateral model automatically splits unless entrained by bright light/dark cycles, or unless it has excitatory inter-hemispheric connections. This suggests that excitatory cross-connections may be important for freerunning behavior. We found that constant light of varying intensities transitions the model between split and unsplit activity only in very limited conditions, but the strength and polarity of the contralateral connections play a much greater role in this dynamical transition. These findings suggest that splitting may involve plasticity of the inter-hemispheric connections of the SCN.

1Classic findings from neuropsychology and animal studies established the hippocampus as a key substrate for rapid learning and episodic memory, with the dentate gyrus exhibiting extreme sparse coding. Sparse coding has long been hypothesized to enable fast learning through pattern separation, enabling rapid separation of highly similar inputs. However, prior computational work has largely focused on episodic memory or simplified linear tasks, leaving open how hippocampal sparsity affects learning speed and generalization in complex tasks. Here, we present a systematic investigation of sparse coding in deep neural networks varying the sparsity level and location (layer depth) and evaluated the functional consequences for learning and generalization. We found that learning performance is maximized at a balanced sparsity level of [~]5%, matching empirical estimates of the hippocampal sparse code. Dimensionality and representational similarity analyses revealed that sparse layers promoted orthogonalization of input representations, mirroring hippocampal pattern separation that enables fast learning. Furthermore, sparsity in early layers led to fast learning only on the training set and poor generalization to a held out test set, reflecting memorization, while sparsity in later layers consistently aided generalization, providing implications for theories of hippocampal-cortical learning. Our findings demonstrate the power and tradeoffs of the hippocampal sparse code, and show how hippocampal-cortical circuits possess the computational capacity to support both fast learning and generalization, depending on where sparsity is implemented. We offer a unifying perspective on how the hippocampus works as a fast, sparse memory system and the hippocampal-cortical pathway as a mechanism for generalizable learning.

Structural connectivity (SC) data are crucial for brain network analysis, but SC-based machine learning often suffers from limited data availability, hindering model generalization and robustness. Although data augmentation using deep generative models has attracted increasing attention, it remains unclear how different models capture the complex topological features of SC data. To clarify the learning characteristics of deep generative models for SC generation, this study compares three representative models: variational autoencoder (VAE), Wasserstein GAN with gradient penalty (WGAN-GP), and denoising diffusion probabilistic models (DDPM). We systematically evaluated these models using both synthetic datasets with known characteristics and real-world SC data. Generation quality was assessed using graph-theoretic metric comparisons and visual inspection of the generated adjacency matrices. WGAN-GP showed relatively stable performance across datasets and metrics, without severe performance degradation across evaluation settings. In contrast, VAE and DDPM performed well in specific aspects but were more sensitive to data characteristics. These findings suggest that WGAN-GP may serve as the most balanced baseline for future SC data augmentation studies, whereas VAE and DDPM may be useful depending on the target application and structural properties of interest. Furthermore, because all models struggled to fully reproduce strict global constraints such as planarity, our results suggest that standard generative models may be insufficient to capture the complex topological features of SC data. This highlights the importance of incorporating the desired structural properties into the training or generation process.

Biological and robotic systems must solve two related computations to move: inverse dynamics, which determines the forces or torques needed to produce a desired movement, and forward dynamics, which maps applied forces to motion. Although these computations are coupled by the same equations of motion, they are usually estimated or implemented as distinct inverse and forward mappings, in both model-based and data-driven formulations. This separation can obscure the shared structure that constrains both problems. Here, we present ANNet, a physics-informed neural network that places both computations on a common learned representation by learning a single scalar quantity from classical mechanics--Appell acceleration energy. The network maps kinematic state and candidate accelerations to this scalar function, and inverse dynamics is obtained by differentiating the learned energy function with respect to acceleration to recover joint torques. Forward dynamics is then calculated without retraining by embedding the same learned energy landscape in an optimization objective whose unconstrained minimum satisfies the Gibbs- Appell equation. The resulting accelerations are integrated forward in time. We evaluate ANNet on a double pendulum paradigm. In trials unseen by the network during training, inverse and optimization-based forward simulations are real-time accurate. Our results provide a first-principles route for using a single learned representation to support both prediction and control. SignificanceRobots and animals must solve two problems to move: computing the forces or torques needed for a desired motion (inverse dynamics) and determining the motion produced by applied forces (forward dynamics), which are usually modeled separately. We show that both problems can be expressed using a single scalar function from classical mechanics, Appell acceleration energy. A neural network trained so that the derivative of this learned function matches reference joint torques performs inverse dynamics. The same network then computes forward dynamics by minimizing an objective built from the learned energy landscape, without retraining. This framework provides a unified representation for prediction and control in both neuroscience and robotics.